Image acquisition system

ABSTRACT

An image acquisition system for machine vision systems decouples image acquisition from the transmission of the image to a host processor by using a programmable imager controller to selectively disable and enable the transmission of data to the host and by using a system of buffers to temporarily store image data pending allocation of memory. This enables the image acquisition system to acquire images asynchronously and to change the exposure parameters on a frame-by-frame basis without the latency associated with the allocation of memory for storage of the acquired image. The system architecture of the invention further permits interruption and resumption of image acquisition with minimal likelihood of missing data. Data throughput is further enhanced by transmitting to the host only that data corresponding to the region of interest within the image and discarding the data from outside of the region of interest at the camera stage.

RELATED APPLICATIONS

The current application is a continuation of Ser. No. 10/355,717, filedJan. 31, 2003, which, in turn, is a continuation of Ser. No. 09/932,275filed Aug. 16, 2001, which, in turn, is a continuation of Ser. No.08/884,589 filed Jun. 27, 1997, now U.S. Pat. No. 6,282,462 which claimspriority to U.S. Provisional Application No. 60/038,690, filed on Feb.7, 1997 and the commonly-owned and U.S. Provisional Application60/020,885, filed on Jun. 28, 1996. The entire contents of each of theseapplications and patents are hereby incorporated herein by reference.

BACKGROUND OF THE INVENTION

The present invention relates to image acquisition systems, and moreparticularly to image acquisition systems suitable for machine visionsystems that capture and process optical images.

Conventional image acquisition systems have been used for decades toacquire and process optical images. The conventional systems typicallyemploy one or more video cameras that acquire an image and frame grabberboards that store and/or process the image. These systems are relativelyeasy to connect since the components of the system are wellcharacterized, readily understood, and result in predictableperformance.

Today's modern manufacturing and industrial installations are becomingincreasingly automated to increase quality and reduce costs to bettercompete in the global market. Consequently, these installationstypically employ machine vision systems which are used to monitorproduction processes, position selected components, and to perform otherimportant manufacturing tasks. There is, however, a mismatch between therequirements of modern machine vision systems and the design goals ofexisting imaging technology typically used with machine vision systems.

Conventional image acquisition systems are designed to provide either acontinuous flow of images or a predictable flow of images at a fixedrate determined by human awareness and response times, and at the lowestpossible cost. With a new frame being transmitted every 1/60 second,missing lines and/or frames in such a system are tolerable because theyare generally unnoticed by a human viewer. The fixed rate of imagecapture and delivery in conventional image acquisition systems is justfast enough to seem infinitely fast or seamless to human perception.Because errors that occur during image acquisition and transfer, such asdropped lines and frames, are only presented to a viewer for 1/60th of asecond, and because the human eye acts as a time integrator, theseintermittent errors are rarely perceptible to a human observer.

On the other hand, machine vision systems often make decisions based ona single still image. Because of this, machine vision systems requireconsiderably higher resolution, contrast, and tolerance for dataintegrity than conventional image acquisition systems. For example, ifthe machine vision system is looking for normal process variations inmanufactured parts, any imaging errors can result in the inappropriaterejection of a part. This leads to unnecessary part waste and anincrease in the total costs of production.

Errors in an image acquisition system include imaging errors andprocessing errors. Processing errors, such as dropped lines and frames,arise from errors that occur as image data travels from the camera tothe host computing system. Imaging errors arise in the camera itself orin the environment outside the camera. The former we refer to as sensorerrors and the latter we refer to as scene errors.

Sensor errors typically arise from differences between photosensitiveelements, (e.g., photosites in a charge-coupled device) that form partof the image acquisition system. As a result of these differences, apair of photosensitive elements may respond differently to the samelevel of illumination. These differences arise from normal processvariations in the manufacture of the photosensitive elements or fromtemperature differences between otherwise identical photosensitiveelements.

Scene errors are those errors that arise from incorrectly illuminating ascene. Since ambient lighting conditions cannot be controlled to thesame level of precision as that resolvable by a modern machine visionsystem, subtle variations in lighting levels or colors, due, forexample, to aging of light sources, can degrade the performance of themachine vision system.

Unlike conventional image acquisition systems which obtain images eithercontinuously or at regular intervals, machine vision systems can requireimages at unpredictable times. For example, in an automated inspectionline, the objects to be inspected may not be spaced apart regularlyenough on the conveyor belt to permit periodic image acquisition.Consequently, a machine vision system does not have the luxury ofknowing, in advance, when an image is to be received.

A machine vision system should also be ready to acquire an image almostimmediately upon request. For example, in the automated inspection linedescribed above, if an object to be inspected is about to enter thecamera's field of view, it is preferable that an image be acquiredrapidly, before the object leaves the camera's field of view.

Because machine vision systems often have to obtain multiple images inrapid succession, a need exists to rapidly transmit image data to aprocessor of a host system. Prior art machine vision systems generallyinterrupt the host processor to obtain a starting memory address inwhich to place an image or portion thereof before the actual imageacquisition and transfer can begin. In such systems, referred to as“software scatter/gather” systems, the host processor plays asignificant role in image acquisition and transfer. This reliance on ascarce resource such as the host processor results in latency periodsduring which image acquisition and transfer cannot occur because thehost processor is busy performing other tasks. These latency periods ofuncertain duration make it difficult, if not impossible, for a machinevision system to repeatedly acquire an image on demand. When the latencyperiod becomes excessively long, collisions can occur in the data pathas data from subsequent images arrives faster than data from previousimages can be processed. This results in lost or erroneous data.

In other machine vision systems, referred to as “hardwarescatter/gather” systems, data transfer is performed by a direct memoryaddress technique (DMA). Although these systems do not require theassistance of the host processor to access memory, they typicallyrequire that a memory segment be dedicated to their use. Thedisadvantage of this method is that the dedicated memory becomesunavailable for use by other processing tasks, even when it is not beingused for image transfer.

In systems of this type, data collisions, as described above, can beameliorated by dividing the dedicated memory into two blocks. Thisenables the system to place an incoming image into the first memoryblock while processing the image in the second memory block. Once thesystem finishes processing the system in the second block, it can beginprocessing the image in the first block, thereby freeing the secondblock to receive another incoming image. A disadvantage of this systemtype is that, when used in conjunction with a multithreaded computingenvironment, the complexity of programming this task becomes rapidlyunmanageable.

It is also desirable for modern machine vision systems to autonomouslydetermine whether or not an image should be acquired. For example, in aninspection line, the objects to be inspected may not be spaced atregular intervals. This raises the problem of how to acquire an imageonly when an object to be inspected is in the camera's field of view orin a particular region within the camera's field of view.

Prior art systems attempt to solve this problem by triggering the camerawith an external sensor located outside of the machine vision system.These sensors, however, are typically difficult to interface reliablywith the machine vision system. Moreover, latency associated with themachine vision system can make it difficult to reliably position theobject to be inspected in the correct region of the camera's field ofview.

Further important constraints imposed by conventional image acquisitionsystems include the relatively fixed field of view, relatively fixedfrequency of image capture, fixed and relatively low rate of imagetransfer to host computer memory (or relatively expensive transfer ofimage), and lack of data integrity, arising, for example, from grayscale errors due to several causes including pixel jitter and skew, orfrom lost data such as dropped lines and frames.

Additionally, current machine vision systems are relatively expensive toinstall and operate. Another drawback of these systems is that they areunable to change the form of the acquired image data in real time, inother words, between each acquired frame or shot.

Modern machine vision systems have been developed to address some ofthese drawbacks. One example devised to ameliorate some of thesedrawbacks includes the use of expensive custom application-specificcircuitry to provide high fidelity and low error image acquisition.These custom systems are typically very expensive to acquire and verydifficult to integrate with existing machine vision systems.

There thus exists a need in the art for an image acquisition systemsuitable for use with modern machine vision systems that is flexible andprovides for high fidelity asynchronous image acquisition and transfer.

SUMMARY OF THE INVENTION

The image acquisition system of the present invention eliminates theseand other sources of image acquisition errors by integrating most of theimage acquisition components into one dedicated machine architecture.This dedicated architecture can be utilized to perform preliminary imageprocessing operations in real-time such as correcting each acquiredimage for both hardware errors and scene errors, recalibrating thesensor array in real time to correct for errors due to differencesbetween pixels or errors in scene illumination, or otherwise spatiallyfiltering the image in real-time, all without burdening the hostprocessor.

A system according to the invention includes an image acquisition stagefor acquiring at least a region of interest from an image in response toa trigger signal. The trigger signal can incorporate exposureinformation for the image acquisition stage and instructions forspecifying the region of interest. This information can be changed on aframe-by-frame basis, in real time and on the fly.

The system then transfers either all or part of the data representingthe image to the host processor by way of a sequence of temporarybuffers. These buffers enable the system to decouple the process ofimage acquisition from image transfer, thereby enabling the system toacquire images without having to wait for the host processor to allocatememory for storage of the system. The sequence of temporary buffers alsoenables a system according to the invention to interrupt the process oftransmission either between images or in the middle of the image and toresume transmission with little likelihood of data loss.

The data throughput for a system according to the invention iscontrolled by means of a programmable imager controller which can drivethe transfer of data from a CCD array or other solid state imagingdevice to the host processor at variable rates in response to the stateof the buffers and in response to instructions from the host processor.This programmable imager controller further increases system throughputby transmitting to the host processor only data from within the regionof interest and discarding data from outside the region of interest.Since data can be discarded more quickly that it can be transmitted,this increase in throughput can be substantial when the region ofinterest is much smaller than the overall image.

The system of the invention can also control selected system parametersduring the acquisition of one or more images. These parameters includethe time and duration of exposure, the particular region of interestwithin the field of view, the particular mode of operation of thesystem, and other parameters that define the framework for imageacquisition and which would be obvious in light of this disclosure toone of ordinary skill in optical and electrical engineering. Asignificant advantage of the present invention is that these parameterscan be changed in real time, between shots or frames, withoutsacrificing bandwidth. This feature allows the system to dynamicallyrespond to requests during the image acquisition process.

The system also enables images to be acquired asynchronously andvirtually on demand without the need to wait for the availability ofsystem memory to store the image. The system achieves this by separatingthe acquisition of an image by the camera, a task of short andrelatively predictable duration, from the task of transmitting the imagefrom the camera to the host processor, a task having an unpredictableand potentially long duration. Consequently, the image capturing stagecan be performed independently of the image transfer stage. Whennecessary, the acquired image can be stored temporarily in memory, suchas in a data FIFO register, while the system waits for a memory addressto place the image into. Because of this separation between imageacquisition and image transfer, the system of the invention can acquirean image without the need to await a memory address in which to placethe image. The system is thus not hampered by the latency associatedwith conventional image acquisition systems. Additionally, the systemcan thus process requests for an image without requiring the destinationaddress in advance.

When the data FIFO register approaches its capacity, it can assert aninterrupt to halt image acquisition as described below. This is achievedby interposing a feedback loop between a camera and the imageacquisition board. This feedback loop enables the acquisition board totemporarily and immediately halt image acquisition and transfer wheneverthere is too much data traffic to permit the reliable transmission ofdata. According to one practice of the invention, data already acquiredby the camera is temporarily stored in the camera throughout theduration of the interruption. By incorporating this function into thededicated architecture, the present invention relieves the hostprocessor from burdensome data management tasks. This provides for anincrease in image fidelity (data integrity) with a correspondingdecrease in the occurrence of errors when acquiring images as well as anincrease in overall image throughput.

An additional feature of the present invention is that interruption ofimage acquisition and transfer, as described above, can occur either atthe end of a frame, at the end of a line within an image, or at the endof any preselected section of an image. The flexibility achieved bypermitting the transfer of image data in units smaller than the entireimage enables the system to take advantage of small gaps in data trafficthat it would otherwise be unable to use, as well as enables the systemto rapidly acquire and transfer image data.

Another feature of the invention is that the system acquires datasignificantly faster than prior art systems, including hybridconventional systems employing conventional imaging and machine visionsystems. According to one practice of the invention, images can beacquired and transferred to the system's image signal processor up to 30times faster than conventional systems, while providing for flexible,high speed control and transfer of the image data.

Unlike prior art systems in which a portion of memory is dedicated tohold the image to be processed, the system of the present inventionexploits modern operating systems' ability to perform dynamic memoryallocation. This permits the system to allocate only as much memory asis necessary to process an image and to allocate that memory only whenit is necessary to do so and on an image-by-image basis. Once the imageprocessing task has been completed, memory allocated to that task can bereleased for use in other processing tasks.

According to one aspect, the system acquires images asynchronously andindependently of the host processor.

This is facilitated by the separation of image acquisition and transfer,thereby permitting acquisition to occur before memory to store the imageis made available. The system is also configured to place a time-stampon the acquired image. According to one practice, controller hardware ofthe host device monitors the system for a camera trigger signal, whichtriggers the camera. This avoids the latency associated with having thehost processor monitor and time stamp the image.

The present invention also provides a structure for autonomouslydeciding, based on an image in the camera's field of view, whether ornot to acquire an image. The system of the invention accomplishes thisby designating a trigger region within the camera's field of view andprocessing the image from the trigger region to determine if the imagein a region of interest within the camera's view should be acquired. Thesystem processes the portion of the image within the designated triggerregion independently of the host processor and at relatively high samplerates.

The foregoing control of the image acquisition process is thuscontrolled on an image-by-image basis in a dedicated architecture thatis smaller and significantly less costly than systems heretofore known,thus dramatically reducing the overall cost of the image acquisitionsystem. Additional features of the invention which aid in the reductionof cost include the simplification of the data paths.

The image acquisition system of the present invention includes an imageacquisition element, such as a camera, for acquiring an image and aprogrammable control element which selectively and programmablyinitiates the performance of a number of selected functions by the imageacquisition element. These functions can include the initiation andtermination of image acquisition, the selection of a particular regionof interest within the acquired image, calibration or filtering of theoutputs of the photosensitive elements that form part of the imageacquisition system, definition of a trigger region, or the purging ofcharge from the photosensitive elements. The image acquisition by thecamera can also be interrupted, in real-time, such that at least aportion of the image is temporarily stored in the camera. Thisinterruption sequence allows the image acquisition system of theinvention to process any previously transferred image data in highlyreliable manner.

The system can be mounted on an acquisition board that functions as aninterface between the camera and a conventional host computing system.The acquisition board manages the transfer of image data between thecamera and the host computing system, where the image data is ultimatelyprocessed.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features and advantages of theinvention will be apparent from the following description and theaccompanying drawings in which like reference characters refer to thesame parts throughout the different views.

FIG. 1 is a schematic block diagram of the image acquisition system ofthe present invention.

FIG. 2 is a more detailed schematic block diagram of the imageacquisition system of FIG. 1.

FIG. 3A-3B are schematic depiction's of the vertical and horizontalregister arrays of the camera component of the acquisition system ofFIG. 1.

FIG. 4 is a more detailed schematic depiction of the image acquisitiondevice of FIG. 1.

FIG. 5 is a more detailed schematic depiction of the camera of FIG. 2.

FIG. 6 is tabular depiction of the states of selected gates of theprogrammable imager controller of FIG. 5 during selected modes ofoperation.

FIG. 7 is a flow chart state diagram illustrating the operation sequenceof the programmable imager controller according to one mode ofoperation.

FIG. 8A is block diagram of the pixel sensitivity correction unit usedto calibrate or filter the output of the photosensitive elements thatconstitute the image acquisition elements.

FIG. 8B is a block diagram of a pixel sensitivity correction unitconnected to the digital input of the A/D converter component of theacquisition system of FIG. 1.

FIG. 9 is an illustration of the trigger region and a region of interestwithin the camera's field of view.

DESCRIPTION OF ILLUSTRATED EMBODIMENTS

FIG. 1 is a schematic illustration of the image acquisition system 10according to the teachings of the present invention. The illustratedsystem 10 includes a central or host computing system 12. The hostcomputing system can be any conventional computing apparatus and cancomprise a display monitor and a dedicated signal processor or can be aclient signal processor which forms part of a larger area network, suchas a LAN or WAN. The host computing system 12 is preferably incommunication with an acquisition board 14, which in turn is coupled toone or more external image capturing devices 16, e.g., cameras. Theacquisition board 14, although illustrated as being located outside ofthe host system 12, can form part of that system as will be appreciatedby those of ordinary skill. Consequently, occasionally the combinationof the two will be referred to as the host system. According to apreferred practice, the acquisition board is similar to a PCI bus cardwhich interfaces with the PCI bus of the host system 12, according tothe teachings of the present invention.

Those of ordinary skill will recognize that a single acquisition board14 can be used to operate a number of cameras 16. Conversely, a separateacquisition board 14 can also be used for each camera 16 of theacquisition.

The illustrated cameras 16 each include a programmable imager controllerstage 20 which is coupled to an image capturing element 24. Theacquisition board 14 preferably generates control signals which aretransferred to the programmable imager controller 20 along communicationpath 28. The output of the camera element 16 is transferred to theacquisition board along data output path 30 for storage and/orprocessing by the host system 12.

FIG. 2 is a more detailed schematic depiction of the image capturingelement 16, acquisition board 14 and host computing system 12 of thepresent invention. The illustrated host computing system 12 preferablyincludes a bus 100, a display controller 106, a memory bus 110, a memorymodule 116 representing actual physical memory, a host processor 120, aco-processor 124, and a virtual memory block 128 that illustrativelystores selected executable and dynamic link libraries and code,designated as code modules 132-138, as well as selected memory for theimage data regions, designated as destination addresses 142-146. Thoseof ordinary skill in computer and electrical engineering will readilyunderstand the operational relationship between the memory, bothphysical and virtual, and the software resident on the host computingsystem 12.

The bus 100 preferably interfaces and communicates with the acquisitionboard 14 and provides structure that allows for the flow of image databetween the camera 16 and the host computing system 12. The term “bus”is intended to include any suitable data signal transmission path, andis preferably a high bandwidth data transmission channel, such as a PCIbus. The display controller 106 preferably communicates with both thebus 100 and the memory bus 110 to allow the controller 106 to displaythe acquired image data on the display monitor 12A, such as a VGAmonitor, and to access stored instructions. The illustrated co-processor124 is preferably coupled to both the bus 100 and the memory bus 110 andserves to direct the acquired image data. The use of this IO controller,i.e. “bridge chips” reduces the need to interrupt the host processor 120to process the image data, and thus decreases the overall processingtime and increases the processing rate of the image acquisition system10.

The image acquisition system 10 can operate, according to one practice,in response to externally applied triggers or in response to signalsgenerated internally by selected software code stored in the memory 128of the central system 12. Specifically, selected imaging modes ofoperation can be rapidly chosen by initiating a request to the system10. The term “rapidly” is intended to mean initiating the request inless than or equal to about 32 ms, and, preferably in less than or equalto about 1 ms. For example, a request can be made by the camerainterface executable program, referred to as cam.exe 132, or by selectedclient software (for example an image analysis or machine visionapplication program), referred to herein as client.exe 134. Theexecutable file 134 thus uses the image acquisition system 10 as asource of image data.

The terminology used herein is appropriate for systems installed incomputers employing Microsoft's Windows 3.11 and DOS v. 6.x operatingsystems. When used with other operating systems, for example Windows NTor VME systems, similar terminology will apply. The structure of thesoftware code designated by cam.exe, client.exe, and the dynamic linklibraries can be easily constructed by the ordinarily skilled computerand electrical engineer by reference to the description of the operationof the image acquisition system 10 of this specification.

According to one practice, when the host computer loads or boots theexecutable program cam.exe 132, a selected environment is established inthe host computer that the camera system employs to service requestsfrom either a user, through the computer's keyboard and mouse, or froman autonomous executable code block, such as the machine vision programclient.exe 134. At a minimum, the program incorporating an interruptcode (interrupt service routine) is loaded into the memory 128 of thehost computing system 12. The role of the interrupt code is to rapidlynotify either cam.exe 132 or client.exe 134 that an image buffer hasbeen filled with the data it requested and is ready for viewing and/orprocessing by the requester. If the program cam.exe 132 needs to responddirectly to user requests, it calls selected dynamic link libraries(DLL's). These DLLs, such as cam.dll, are typically used to storeinstantaneously accessible lists of functions and resources or toallocate virtual and physical memory for image buffers located in thehost computing system 12. Alternatively, when the program client.exe 134issues requests for an image directly, it allocates sufficient bufferspace in memory by making direct calls to cam.dll 138.

Camera Setup

In all modes of operation, either cam.exe 132 or client.exe 134establishes the operating framework for the image acquisition system 10by providing selected camera setup information, including the particularmode of camera operation, the particular region of interest, and properexposure times, and by providing host memory setup requirements,including image buffers to receive, hold and process the image data forone or more cameras 16. These programs can thus be used to generate thecamera instructions stored in the camera setup store 48 and reservedphysical addresses for image data in the destination address store 90,as discussed further below.

As used herein the term “region of interest” is intended to include aregion or portion of an acquired image that is smaller in one or morespatial or axial dimensions than the entire image acquired by the imageacquiring device or system. The region of interest is preferablyselectable. Additionally, those of ordinary skill will recognize thatthe programs can establish multiple and different regions of interestwhich are loaded into the camera setup store 48 and the destinationaddress store 90, as described in further detail below. These multipleregions of interest allow the system to dynamically acquire an image andprocess different selected portions of the entire acquired image.Additionally, these multiple regions of interest allow the system totrigger acquisition of an image from one region of interest based on thecontent found in another region of interest.

According to one preferred practice, the region of interest can bespecified by providing the line and pixel number of the upper leftcorner of a rectangular region in the image and similar information forthe lower right hand corner of the region. The filled image buffersestablished by the host operating system can be areas of contiguousvirtual memory that are reserved and mapped to physical regiments inselected and variable block sizes, e.g., 512 byte to 4K byte blocks, andthat have a defined initial physical memory address, denoted as memorylocations 142-146. Those of ordinary skill will appreciate that imagebuffers contiguous in virtual memory space may be realized bynon-contiguous locations in physical memory space. It is anticipated bythe teachings of the present invention that only within each physicalblock are physical addresses contiguous. This facilitates theasynchronous mixing of data into one DMA channel from multiple sourcesduring multiple camera acquisitions.

All of the foregoing information is communicated to the camera 16, tothe acquisition board 14 and to the host computing system 12 in the formof a call, in a language specified for and compatible with the imageacquisition system, to a selected code module, for example, cam.dll 138.The selected code module responds by making API or similar calls to theoperating system of the host computing system 12 to allocate one or moreregions of contiguous virtual memory, typically in sizes ranging between1-300K byte and larger. Each such region of contiguous virtual memory isdivided into subsets, typically 1-4K byte blocks, that can be a sizeconvenient for the operating system. The operating system is thenqueried for the physical address of the first byte in each of thesesubsets. The foregoing physical addresses are communicated, via API(application programming interface) or similar calls, to the cameradestination address store 90. In a DOS/Windows/PCI environment, eachsubset is itself contiguous in physical memory. The program cam.dll 138transfers this information into on-board memory, which directly controlsthe cameras and which can reside in, among other locations, aninput-output bus 102 (typically a PCI bus). The beginning addresses ofthe image buffer physical memory block for each image buffer are thenstored in a selected memory location, designated as the cameradestination address store 90. Meanwhile, the exposure times, cameramodes, and regions of interest are also loaded into the camera setupstore 48.

Image data is transferred to the foregoing memory allocated for imagebuffers via direct memory address (DMA) transfers that typically do notrequire processing by the host processor 120. Hence, the host processor120 need only be interrupted upon completion of an image transfer. Thoseof ordinary skill in computer and electrical engineering will appreciatethe type of code that can be employed to perform the foregoing andfollowing actions.

At the same time that it is transferred to host physical memory, imagedata can also be directed, via DMA transfer, to a display controller106. A look up table (LUT) 96 converts pixel data in real time into aform appropriate for the host computing system 12 and its displaycontroller 106.

Initiating Image Acquisition

With further reference to FIG. 2, the acquisition board 14 includes anexternal trigger interface 34 having a plurality of external triggerinputs 36 to accept signals generated by a variety of external sources,including the host system 12. Upon receiving an appropriate signalthrough the trigger input 36, the external trigger interface 34generates trigger output signal 38 instructing the camera loader 42which camera or array of cameras to use.

The illustrated camera loader 42 has multiple inputs, e.g., supports ninputs, and generates one or more camera trigger signals 44 that driveone or more cameras 16 designated by the particular input signal. Thecamera trigger signal 44 preferably includes selected camera setupinformation, including but not limited to the startline and endline ofthe region of interest (ROI), exposure time of the image to be acquired,and mode of operation. The particular mode of operation of theillustrated system 16 designates the operational sequence and parametersof the camera 16. Representative modes of operation are described ingreater detail below. This camera setup information is typicallycommunicated between the host processor 120 and acquisition board 14 bythe PCI bus 100. The camera loader 42 further communicates with selectedmemory blocks, illustrated as camera setup storage block 48 andproximate register storage block 52, the functions of which aredescribed in further detail below.

The host processor 120 initializes the image acquisition system 10 bystoring selected setup information. This set up information includesmode of camera operation, regions of interest within the image, andexposure times. The host processor 120 further provides additional setuprequirements for the camera 16. These include image buffers that aparticular client uses to receive, hold and process image data. The hostprocessor can select a particular region of interest within the imagecaptured by a camera 16 by specifying the line and pixel number of theupper left corner of the rectangular region of interest, and byspecifying similar information concerning the lower right-hand corner ofthe region of interest. The host computing system thus defines,according to simple calls, the region of interest for a selected frame.This selected region of interest can remain constant or can be changedeither after each frame or after a selected number of frames.

Image Acquisition

Referring again to FIG. 2, the camera trigger signal 44 generated by thecamera loader 42 for a particular camera 16 is loaded into theprogrammable imager controller 20 of that camera 16. The camera triggersignal 44 can be transmitted to the camera at any time and without theneed to have a destination address already allocated for the image to beacquired. If the image is acquired before a destination address can bemade available, the image data can simply wait at one or more locationson the data path, such as in a data FIFO 94, until a destination addressbecomes available, as described below.

According to the illustrated embodiment, the camera trigger signal 44for a selected camera 16 is generated by the camera loader 42 inresponse to the instructions stored in the camera setup store 48 forthat camera and in response to the output signal 53 of the proximityregister store 52. Specifically, the camera loader 42 accesses theinstructions stored in the camera setup store 48 for the selected camera16 and shifts the appropriate bits to the camera. The camera triggersignal 44 is preferably a signal that includes: w bits designating theexposure time, x bits designating the first line of the region ofinterest, y bits designating the number of lines in the region ofinterest, and z bits designating the camera mode for this shot. Thenumbers w, x, y and z depend on the CCD array circuit 54 of the camera16.

The camera trigger signal 44 can be different for each imageacquisition. As a result, it is possible to change the contents of thecamera setup store 48 at each frame. This enables the machine visionsystem to change exposure time, region of interest, or camera modebetween frames. Because image acquisition can proceed independently ofimage transfer, a change in the contents of the camera setup store 48has no appreciable impact on system performance.

The processes of image acquisition and memory allocation for the imagecan be performed in parallel with and substantially independently ofeach other. This enables the system's memory manager to generatedestination addresses as system requirements dictate. Among theadvantages of this feature are that it enables the system: to acquire animage almost immediately after a request for an image is issued andwithout the often unpredictable latency associated with waiting for adestination address; to queue several image acquisition requests; and toperform time-consuming image processing tasks, for example writing it toa disk, independently of image acquisition and transfer. Hence, imageacquisition can be initiated without regard to the allocation of adestination address, which is eventually stored in memory 90.

In one instance, the programmable imager controller 20 preferablyincludes a programmable logic device such as a field programmable logicarray circuit (FPLA) and a voltage converter for transforming selectedsignals into a form compatible with the remaining camera components.Those of ordinary skill in electrical engineering and circuit designwill understand that the FPLA is an ASIC chip that can be designed tooperate in a manner in accordance with the teachings of the presentinvention.

The illustrated programmable imager controller 20 is programmable inthat the receipt of a different camera trigger signal 44 at any selectedtime, e.g., after each frame exposure or after any selected number offrame exposures, initiates a different image capturing scheme. Forexample, the camera 16 can be instructed, via the programmable imagercontroller 20, to transmit a different region of interest after eachframe. This provides for a relatively simple method of dynamicallycontrolling the region of interest and exposure without requiring theuse of complex image capturing and processing circuitry. Furthermore,this programming is performed relatively rapidly. For example, in lessthan about 12 μs and preferably less than about 1 μs, the programmableimager controller 20 can generate a new set of instructions to the CCDarray circuit 54.

According to a preferred practice, the arrival of the last bit in theprogrammable imager controller 20 arms the camera 16. subsequently, thecamera loader 42 generates a camera trigger signal 21 that istransferred to the image capturing portion of the camera 16, whichincludes a CCD array circuit 54, an image processor 60, and anoscillator 58 local to the camera. The illustrated camera oscillator 58preferably transmits a camera timing signal 58B to the programmableimager controller 20 and to the acquisition board 14. This camera timingsignal 58B synchronizes the acquisition board 14 with the camera 16 andwith the host computing system 12. Those of ordinary skill in computerand electrical engineering will recognize that the camera triggersignals can be loaded in other ways.

Once the programmable imager controller 20 receives the camera triggersignal 21, the CCD array circuit 54 initiates the exposure processaccording to the teachings of the present invention. This exposureprocess preferably lasts for a selected exposure period, as defined bythe selected exposure bits which comprise part of the camera triggersignal 44 generated by the camera loader 42.

The system 10 runs kernel level software which is responsive to thepresence of the camera trigger signal 21 or a trigger signal on theexternal trigger inputs 36. Upon the occurrence of either of thesesignals, the kernel level software requests the system time from thehost processor 120. This request is set to have priority sufficientlyhigh to ensure negligible latency in the response of the host processor120. The system time is then made available for associating with theimage data acquired in response to the trigger signal.

Transmitting the Acquired Image Out of a CCD Array

The programmable imager controller 20 receives a camera trigger signal21 representative of selected image capturing parameters. This signal isreceived by the CCD array circuit 54. The receipt of this trigger signalinitiates a sequence of signals for the control of a solid state imagersuch as a CCD array. The mechanism used by the CCD array to deliver animage does not affect the operation of the invention. The illustratedCCD array circuit 54 in the preferred embodiment is an interlinetransfer CCD which includes a CCD sensor array having a number ofphotosites corresponding to a selected number of pixel locations and aprogressive scan chip that provides for the shifting of acquired opticaldata into a vertical array of registers. For example, if the camera isdesigned for 640×480 resolution, then there exists a photosite array of640 columns and 480 rows of photosites, i.e. a photosite behind eachpixel. Those of ordinary skill will appreciate that the CCD arrayfunctions as an integrator of light over time, and need not be describedin further detail herein.

Referring to FIG. 3A, in an interline transfer CCD, each photosite 70accumulates charge corresponding to that portion of the image to whichit is exposed. At the expiration of the exposure time and in response tothe imager controller 20, each photosite 70 transfers its stored chargeto a shadow register 80 a in a vertical array of registers 80 associatedwith the column of photosites. This transfer occurs simultaneously forall photosites in the array. Although FIG. 3A shows only two columns ofphotosites and two vertical arrays of registers, those of ordinary skillwill appreciate that for a 640×480 resolution there can exist 640vertical registers.

Referring to FIG. 3B, in a frame transfer CCD, each photosite 70 aaccumulates charge corresponding to that portion of the image to whichit is exposed. At the expiration of the exposure time and in response tothe imaging controller 20, each photosite 70 a transfers its storedcharge to an adjacent photosite 70 b. The photosite at the edge of theCCD array 70 g transfers its charge to the topmost register 80 a in thevertical array of registers 80. This procedure is repeated until thecontents of the topmost photosite 70 a in a column of photosites 70 hasbeen shifted into the topmost register 80 a in the vertical array ofregisters 80.

It is apparent that the net result in both the frame transfer CCD ofFIG. 3B and the interline transfer CCD of FIG. 3A is identical, namely avertical array of registers 80 in which each register contains a chargecorresponding to the charge held by a corresponding photosite.

The CCD array circuit 54 can further include a horizontal array ofregisters 82 having as many registers as there are vertical arrays ofregisters. Hence, for 640×480 resolution as in the example above, thehorizontal array of registers 82 would include 640 registers. Theregisters comprising the horizontal array of registers preferablycommunicate with the illustrated substrate surface 84. Although only onehorizontal array of registers is illustrated in the drawing, those ofordinary skill will appreciate that a number of horizontal arrays ofregisters can be used.

With further reference to FIGS. 3A and 3B, once the image has beenacquired by the camera and the charge associated with that image hasbeen transferred to the vertical arrays of registers, the programmableimager controller 20 begins shifting the image data stored within thevertical arrays of registers 80 into the vertical array of registers 82.For a 640×480 image, the horizontal array of registers be horizontallyshifted 640 times before the image transfer is completed.

In a conventional machine vision system acquiring an m×n image, at theend of each of the m shifts, the system 10 cannot decide whether thecurrent line of registers corresponds to a line of the image within theregion of interest. As a result, all the contents of all the registersare transmitted to the acquisition board for further processing,regardless of whether or not the contents of the registers correspond toa line above or below the region of interest. Typically, it takes 50 to100 times longer to discard the contents of the n registers than it doesto shift a row from the n vertical arrays of registers into the nregisters in the horizontal array of registers. As a result, inconventional systems, the rate at which an image can be transferred islimited by the rate at which the horizontal array of registers can beoperated. Because of this, conventional systems fail to exploit thespeed with which multiple lines of an image can be shifted into thehorizontal array of registers.

In the system 10 of the present invention, at the end of each of the mshifts, the system can decide whether the contents of the n registerscorrespond to a line of the image within the region of interest. If theydo, the contents of the n registers are transmitted to the acquisitionboard for further processing just as they were in the conventionalsystem. However, if the contents of the n registers do not correspond toa line of the image within the region of interest, the n registers arequickly overwritten by the next row of n registers from the n verticalarrays of registers. Any excess charge either simply “spills” into thesubstrate 84 with which, as set forth above, the registers are incommunication or is removed in one shift through the horizontalregister. As a result, the horizontal array of registers only has toperform the slow process of transferring data to the acquisition boardwhen a line from the region of interest has actually been loaded intoit. In this way, the present invention is able to exploit the speed withwhich charge from the contents of the vertical array of registers can betransferred to the horizontal array of registers.

The vertical array of registers 80 is shifted until the line ofinformation located immediately before the first line of the region ofinterest, as defined by the control bits generated by the camera loader42, is placed in the horizontal register 82. The next line is thenshifted vertically down, and then clocked horizontally out of thehorizontal register, to remove all charge therefrom.

The camera trigger signal 44 also includes the ending line of the regionof interest. Thus, the image data associated with each line of theregion of interest is clocked into the horizontal register 82 and thenread out. This information is then transmitted to the image processor60, as denoted by CCD array output signal 59 and as illustrated in FIG.2. The image processor 60 conditions the CCD array output signal 59prior to transfer to the acquisition board 14. The image processor 60 iswell characterized and known in the art and need not be describedfurther herein.

The remaining image data, if any, associated with the portion of theimage outside the region of interest is then shifted out of thehorizontal and vertical arrays of registers 80, 82, and discarded. Thisis done by rapidly shifting the data from the array of verticalregisters 80 into the array of horizontal registers 82.

In addition to the method of fast vertical shifting through linesoutside the region of interest, further acceleration is achieved bytrading vertical resolution for speed within the region of interest. Themode word z above includes the number of adjacent lines to be combinedbefore each reading of the horizontal register during transfer of datawithin the region of interest. By this method two or more verticallyadjacent pixels are added together in the horizontal register and readout as one data value.

Operation of the Programmable Imager Controller

FIGS. 4 and 5 illustrate in further detail the programmable imagercontroller 20 and camera 16 of the present invention. As shown, thecamera trigger signal 44 representative of camera control information istransmitted to the programmable imager controller 20, along with anyFIFO status signal 79, described further below, and the camera timingsignal 58B of the camera oscillator 58. In response, the programmablegenerator 20 produces a number of output signals, e.g., SD, V1, V2, V3,XFR, H1, H2, RG, Sh1, Sh2 ClpDm, IPOp, CB, and Cs, as well as an outputsignal that is transmitted to the image processor 60. The illustratedoutput signals communicate with an image processor 60, a CCD powerregulator 152, a vertical driver 156, and a CCD sensor array 160. TheCCD power regulator 152 converts the input signal 152B to a DC outputsignal 152A that communicates with the CCD sensor array 160 and with thevertical driver 156. To preserve data integrity, the charge pumping bythe CCD power regulator 152 is synchronized with the CCD sensor array160 so that the switching transients associated with the DC outputvoltage do not interfere with the transfer, storage or processing ofanalog image data.

The output signals V1-V3 cause the vertical driver 156 to drive orcontrol the vertical shifting of the vertical array of registers 80. Thesignal SD initiates the purging of charge from the sensor array, asdescribed in further detail below. The output signals H1 and H2 drivethe horizontal array of registers 82 during operation.

The operation and use of the illustrated image processor 60, CCD powerregulator 152, vertical driver 156, and CCD sensor array would beobvious to the ordinary skilled artisan in the field of electricalengineering.

With reference to FIG. 5, the programmable imager controller 20 of thepresent invention can be characterized by an illustrated state machine166 connected to a number of serially connected registers 168-174, whichload the camera trigger signals 44 generated by the camera loader 42.The programmable imager controller 20 also includes a shift-in finishedcounter 176, and a shot-finished counter 178, a vertical controlgenerator 180, a horizontal control generator 182, and a CCD imageprocessor signal generator 186.

FIG. 7 illustrates a flow chart schematic diagram of the state machineand associated circuitry of the programmable imager controller 20 of thepresent invention. During operation, the illustrated timing generator 20waits for the next communication packet to be received, as illustratedin step 188. According to step 190, the communication complete counter176 is checked to see if it is equal to zero. If it is not, the timinggenerator 20 continues to receive command data. If the counter 176 isequal to zero, the appropriate counters are loaded with the cameracontrol information 44 designated by instruction bits x, y, z and wasillustrated in FIG. 5. The state machine 166 then produces a SIEN outputsignal 181A which is received by the vertical control generator 180,according to step 194. In step 196, the exposure counter 172 receivesthe vertical shift interval clock signal, and according to step 198, thesystem checks to see if the exposure counter 172 is equal to zero. If itis, the exposure counter 172 is cleared, i.e., is nulled, as illustratedby step 200. The state machine 166 then toggles the signal carried along181B and received by the vertical control generator between a logicalhigh and a logical low, as illustrated by steps 202 and 204. Thereafter,the state machine 166 generates a logic high along 181A, as shown bystep 206.

According to step 208 the shot-finished counter clock 178 receives avertical shift signal. According to step 210, the state machine 166generates logical highs along paths 181C and 181D.

In accordance with step 212, the ROI (region of interest) line counter170 is vertically shifted until this counter is zero, as illustrated bysteps 214 and 216. At this time, the state machine 166 sets the verticalclock's speed to a logical low along path 181D. This is received by thevertical control generator 180 (step 218). The ROI last line counter 168is likewise vertically shifted until the counter reaches zero, inaccordance with steps 220 and 222. The state machine 166 then generatesa logical high along path 181D. The illustrated system then checks tosee if the shot-finished counter 178 is equal to zero. If so, the systemproduces logical lows along paths 181C and 181D, as illustrated by steps228 and 230. The programmable imager controller 20 then reloads theshot-finished counter, in accordance with step 232. Those of ordinaryskill will recognize that the block diagram schematic depiction of theprogrammable imager controller 20 in conjunction with the flow chartdiagram illustrating the operation thereof effectuate the vertical andhorizontal clocking of the registers of the CCD sensor array, as well aseffectuate the purging of charge from the photosites of the CCD array.

FIG. 6 shows, in tabular format, the states of selected gates of theprogrammable imager controller during image acquisition andtransmission.

During image acquisition, the signals SIEN and XFREN cooperate to enableimage exposure. SIEN drops to logical 0 to arm the camera for exposure.Note that SIEN does not trigger the exposure. Exposure begins when XFRENtransitions from a logical 0 to a logical 1 and ends when XFREN returnsto its idle state at logical 0. Upon termination of exposure, SWENreverts back to a logical 1, thereby disarming the camera.

During image transmission, VCKEN transitions to a logical 1 to arm thetransmission of data from the camera to the acquisition board. For linesof image data forming part of the region of interest, the transmissionalternates between a fast shift step during which image data isvertically shifted one line at a time from the vertical array ofregisters to the horizontal array of registers and a readout step duringwhich the line of image data is shifted out of the horizontal array.VCKEN remains at a logical 1 throughout both of these steps.

Switching between the fast shift step and the slower readout step iscontrolled by the signals VCKSP and HCKEN. During the fast shift step,VCKSP is set to logical 1. VCKSP drops to logical 0 to arm the readoutstep. Note that VCKSP does not actually initiate the readout step.Readout begins when HCKEN drops to a logical 0 and ends when HCKENreturns to a logical 1.

Allocation of Memory for Storage of Acquired Image

As set forth above, the process of allocating memory for storage of anacquired image can occur independently of and concurrent with imageacquisition. This memory allocation method, which will be referred to as“hardware scatter/gather,” makes use of a modern operating system'sability to perform dynamic memory allocation.

In hardware scatter/gather systems, the system's memory manager, uponrequest of a software application, will allocate memory for imagestorage. Once the memory is no longer needed for image storage, thememory manager can return the allocated memory to a common memory poolfor use in subsequent computing tasks. In this way, the system canallocate a different memory address to each frame and can allocate onlythe amount of memory necessary at any instant. Since memory allocationcan proceed concurrently with image acquisition, throughput of thesystem is improved and acquisition of an image can be triggered with aminimum of latency.

Dynamic memory allocation algorithms are available in many modernoperating systems. The use of such dynamic memory allocation is thuswell within the capability of one having ordinary skill in the art ofcomputer engineering.

Pixel Sensitivity Correction

With reference to FIGS. 2, 8A, and 8B, according to an optional featureof the invention, the image processor output signal 61 is transferred toa pixel sensitivity correction module 62 which multiplies the outputcorresponding to each photosite 70 by a predetermined pixel sensitivityfactor. This pixel sensitivity factor can compensate for the differingresponse characteristics of each photosite 70 or for errors caused bythe inability to precisely control the scene illumination. Additionally,the multiplication of the output corresponding to each photosite 70 byits corresponding pixel sensitivity factor can result in spatialfiltering of the acquired image. In this way, the pixel sensitivitycorrection module 62 can perform a preliminary image processing step inreal time without interrupting the host processor 120. Since thepreliminary image processing step performed by the pixel sensitivitycorrection module 62 would otherwise have to be performed by the hostprocessor 120, the presence of the pixel sensitivity correction module62 saves overall processing time and increases the throughput of thesystem.

Referring to FIG. 8A, the pixel sensitivity correction module 62comprises memory that stores a pixel sensitivity factor storage table63, which includes the pixel sensitivity factors and a multiplier 64.Additionally, the pixel sensitivity correction module 62 can include acounter 65 which rolls over at the number of pixels in the region ofinterest.

In the illustrated embodiment, the pixel sensitivity correction module62 is connected to the analog side of the A/D converter 74. However, thepixel sensitivity correction module 62 can also be connected to thedigital side of the A/D converter 74, as shown in FIG. 8B.

The output of the pixel sensitivity correction module is connected tothe analog input of an analog-to-digital (A/D) converter 74 whichconverts the analog output signal of the camera to a digital signal. Theconstruction of such converters is well known in the art of electricalengineering and suitable converters are commercially available. In onepractice of the invention, the A/D converter 74 can be a component ofthe camera 16. In such a case, the pixel sensitivity correction modulewould be connected to the digital side of the A/D converter 74 as shownin FIG. 8B.

In operation, the pixel sensitivity correction module 62 accepts datafrom either the image processor 60 as shown in FIG. 8A or the digitalside of the A/D converter 74 as shown in FIG. 8B. In either case, thepixel sensitivity correction module uses its counter 65 to determinewhich entry from the pixel sensitivity factor storage table 63corresponds to the pixel currently at the input to the pixel sensitivitycorrection module 62. The corresponding entry from this storage table isthen made available to the multiplier 64 which multiplies it by thevalue of the pixel currently at the input to the pixel sensitivitycorrection module 62. The product is then transmitted from themultiplier 64 to the analog input of the A/D converter 74 as shown inFIG. 8A and in FIG. 2 or to the data FIFO 78 as shown in FIG. 8B. Inmost embodiments, an adder precedes the multiplier 64. This addermodifies the pixel value by a unique offset correction stored inparallel with the sensitivity correction factors.

Data Transmission from the Data FIFO to the Host Computing System

Each camera 16 in the illustrated system 10 has associated with it adata FIFO 78 in which image data accumulates as it arrives from thecamera 16. This data FIFO 78 is periodically emptied by transmitting theaccumulated data stored within it to the memory location allocated forthat camera 16, the starting physical address and extent of which arestored in the destination address store 90. The data FIFO 78 isgenerally emptied when the amount of accumulated data reaches some dataFIFO threshold. The data FIFO 78 can be emptied of accumulated data attimes independent of the times at which data arrives at the data FIFO.For this reason, the data FIFO makes possible the asynchronous transferof data between the camera 16 and the host computing system 12. As usedherein, the term “data FIFO” is intended to include contiguous andnon-contiguous memory and registers, including FIFO and other memorytypes. The memory or register can form part of the memory of the hostdevice, or can be implemented in SRAM, DRAM, FLASH, or remote drives, oron other memory associated with a dedicated electrical circuit used inconjunction with the camera 16 and the host computing system 12 of theinvention.

This occurs when the acquisition board 14 performs a DMA (direct memoryaddress) transfer of the data to the address locations defined in thecamera destination address store 90. The camera destination store 90 ispreferably pre-loaded with an appropriate address definitions by thehost processor 120 of the host computing system 12. The choice of thisdata FIFO threshold is important for the efficient and economicalfunctioning of the system. If the data FIFO threshold is chosen toohigh, it becomes necessary to use data FIFO's having sufficient capacityto store the data. Such high capacity data FIFO's can be prohibitivelyexpensive. If, on the other hand, the data FIFO threshold is too low,the system 10 will have to frequently access each FIFO, retrieving onlya small amount of data therefrom. This is an inefficient use of systemresources.

In the illustrated embodiment, the image acquisition performed by thecamera 16 can be conveniently interrupted between the end of one line ofthe image and the beginning of the next line. Thus, a convenient dataFIFO threshold is a single line of the image. However, the invention isnot restricted to the use of a single line of the image as the data FIFOthreshold.

The transmission of image data from the data FIFO 78 to the host memory116 proceeds until the last line of data corresponding to the selectedregion of interest has been transmitted. The accumulated data retrievedfrom the data FIFO 86 is transferred to the PCI bus 100 through the businterface 102 and then transferred through the IO manger to host memory,significantly reducing the number of times the host processor 120 isinterrupted, thus increasing the processing speed and efficiency of theoverall system 10. The co-processor 124 also communicates with thememory bus 110. Prior systems utilize the host processor 120 to performeach data transfer thereby requiring that the processor be interruptedfor each data transfer. The present invention overcomes this drawback byusing DMA to move relatively large blocks of image data, therebysignificantly reducing the number of host processor interrupts.According to a preferred practice, the IO manager 120 is interruptedduring acquisition of the region of interest. The software stored in thehost computing system, e.g., cam.dll, calculates the number of pixelsreceived from the region of interest and transmits the trigger thresholdparameters to the proximity evaluator 88. The parameters transmitted tothe proximity evaluator 88 preferably define a summary statistic for aportion of the region of interest whose values are monitored todetermine when the object whose image is to be captured is present inthe camera's field of view. The pixel value contained within proximityevaluator 88 is then compared with a preselected value stored in theproximity register store 52. When this value is reached, the proximityevaluator 88 generates a proximity evaluator output signal 88A that istransferred to the camera loader 42. In response, the camera loadergenerates another set of instructions which are transferred to theprogrammable imager controller 20.

Interruption of Data Transfer

A significant advantage of the image acquisition system 10 of theinvention is that it compensates for the unpredictable delays intransferring image data from the camera 16 through the acquisition board14 to the memory 128 of the host computer. This is accomplished by thereal-time interruption, for a selected period of time, of datatransmission by each camera, before data path overflow occurs. Any dataalready acquired by the camera 16 but not yet transmitted to theacquisition board 14 is stored in the camera's own storage facility,e.g., in the vertical array of registers 80, for the duration of theinterruption period. This provides for a cost-effective method ofstoring the untransferred portion of an image.

Data transmission from the camera 16 can be interrupted as frequently asnecessary. Moreover, interruption can occur not only between frames butat the conclusion of transmission of any preselected subset of theframe. For example, according to one practice, interruption can occurbetween the end of one line and the beginning of the next line. This isadvantageous since data traffic on the host computing system 12 is anunpredictable function of all the activity in the system, much of whichis often unrelated to the transfer of image data. Systems lacking areliable method of managing data traffic can fail to provide the highfidelity data transmission required by machine vision and other imageprocessing systems. For example, prior systems are known to randomlydrop lines or even large portions of a frame of image data. In machinevision applications, this loss of image data can result in improperfunctioning of the system.

With further reference to FIG. 2, while the acquisition board 14 isperforming this DMA data transfer, additional image data continues to beread into the data FIFO 78. This data is bundled together andtransferred to the next address location unless other traffic on thehost computer's I/O bus 100 delays this transfer. In the event that bustraffic prevents the transfer long enough for more than an arbitrarilylarge fraction of the data FIFO to be filled, a data FIFO status bit 79is asserted and transmitted to the camera 16. If the data FIFO statusbit 79 is received by the programmable imager controller 20, theprogrammable imager controller 20 interrupts data transfer byinterrupting the next vertical shifting of data into the vertical arrayof registers 80 and, optionally, by interrupting the read-out of imagedata from the horizontal array of registers 82. This interruption isadvantageous since it ensures that no amount of acquired optical data islost, e.g., a dropped line or frame, due to data traffic in the hostcomputing system. The data FIFO status bit 79 provides for a feedbackloop between the acquisition board 14 and the programmable imagercontroller 20 for sensing data overflow. In response to thisinterruption, the camera 16 retains the data within the CCD verticalregisters 80 for as long as necessary for the host computing system 12to resume accepting DMA transfers. This feedback process results in ahighly reliable and cost effective image acquisition system in which theacquisition hardware itself is used to temporarily store the acquiredimage data. Furthermore, through this interruption feature and itsassociated feedback loop, the invention maintains the integrity of theacquired data and nearly eliminates the loss or corruption of data dueto unpredictable latency periods in performing DMA transfers.

A transfer-complete interrupt is communicated via the host to the clientexecutable when the programmable imager controller 25 signals that allpixels from the region of interest have left the camera head 16, and thevalue of FIFO status_(x) 79 indicates that the Data FIFO is empty. Thisinterrupt is preferably generated after the last byte of image data hasbeen flushed through the system.

Purging the CCD Array

After the transfer gate of the CCD array is restored to its normalstatus, the programmable imager controller 20 sends a continuing set ofpulses to the CCD array circuit 54 to purge the photosites of anyaccumulating charge until the next exposure. This ensures that unwantedoptical data is neither stored nor processed by the acquisition system10. This purging process preferably continues in parallel with otheroperations until the next exposure command is generated by the cameraloader 42 the camera is operating in pipelined exposure mode.

Referring to FIG. 4, to purge the CCD array, the programmable imagercontroller 20 transmits the signal SD to the vertical driver 156. Thiscauses the vertical driver 156 to rapidly shift the contents of thevertical array of shift registers 80 into the horizontal array of shiftregisters 82. This rate at which data is shifted from the vertical arrayof shift registers 80 into the horizontal array of shift registers 82 istypically much faster than the rate at which the horizontal array ofshift registers can be shifted horizontally. However, since purgingoccurs only when the data in the vertical array of shift registers is ofnot interest, the corruption of data in the horizontal array ofregisters is unimportant.

Transmission from the VGA FIFO to the Video Display

The digital data signal 75 of the A/D converter 74 is also transferredto a display FIFO 94. The display FIFO 94 stores the image data forsubsequent processing and display on a display monitor (not shown). Thedigital data signal 75 from by the A/D converter 74 is converted to asignal suitable for the display monitor by the lookup table (LUT) 96.The destination address for the data stored in the display FIFO 94 isstored in the display destination address store 108. The address storeholds the storage address of the memory location to which the acquireddata is to be transferred. As used herein, the term “display FIFO” isintended to include any appropriate memory location that can store datain selected byte sizes.

Another advantage of the present invention is that it does not usephased lock loops. Phase locked loops are prone to timing errors whichcan result in pixel jitter and skew. Consequently, the image acquisitionsystem of the present invention reduces pixel jitter substantially tozero. Moreover, the absence of phase locked loops simplifies the datapath and reduces system cost.

Modes of Operation

The image acquisition system of the present invention has several modesof operation. According to a triggering mode, the image acquisitionsystem can be externally triggered. This can occur when a trigger frommachinery external to both the camera 16 and the host computing system12 sends a signal (typically a signal having an edge) to an externaltrigger interface 34 having n inputs, one for each of the camerasconnected to the system. The arrival of the pulse edge on one of theexternal trigger inputs 36 triggers the camera loader 42 to generate andto transfer a camera trigger signal 44 to the camera 16 designated bythe particular external trigger input 36. The camera trigger signal 44is then transferred to the programmable imager controller 20 for theselected camera 16.

At the expiration of the preselected exposure time, the programmableimager controller 20 initiates a charge transfer from the photosites 70of the CCD array circuit 54 to its vertical array of registers 80. Theimage data representative of the region of interest is then clocked outof the vertical array of registers 80 and into the horizontal array ofregisters 82. Image data from outside the region of interest is clockedout of the vertical registers and into the horizontal array of registers82 at speeds significantly greater than the speed at which data can beclocked out of the horizontal register. This selectively fast “dumping”of unwanted image data allows the system 10 to access and to obtainrelatively quickly the image data corresponding to the region ofinterest. The acquired image data that results from overwriting the datain the horizontal array of registers is ignored.

After the transfer gate of the programmable imager controller 20 isrestored to its normal status, the imager controller transfers a set ofpulses to the CCD array circuit 54 that causes the CCD to continuouslyextract charge building up in the photosites of the array. This processpreferably continues in parallel with other operations until theexposure for the next image begins.

For the number of lines in the region of interest, the programmableimager controller 20 drives the CCD array circuit 54 at a rate thatprovides uncorrupted image data to the computing host system 12. Theimage data is conditioned in the camera 16 by the image processor 60after which it is transferred to the acquisition board 14. Theacquisition board 14 can include a pixel sensitivity correction module62 for multiplying each pixel of the image by a predetermined pixelsensitivity factor and collects the resulting image data in the dataFIFO 78 until the amount of accumulated data reaches a threshold. Theacquisition board 14, in conjunction with the host computing system 12then performs a DMA transfer (a data burst) of the accumulated data tothe address defined in the camera destination store 90. These addressespreferably correspond to the first physical block of memory of thecurrently specified image buffer. Data continues to be read into thedata FIFO 78 even during the DMA transfer.

When the data in the data FIFO 78 again accumulates past the threshold,another DMA transfer is made to the next starting address, unless busloading by other data traffic on the host computer's I/O bus delays thistransfer. In the event that bus traffic prevents the transfer of datalong enough for more than half of the data FIFO to be filled, the dataFIFO status bit is asserted. When this occurs, the data FIFO'sassociated camera suspends data acquisition and transmission and holdsany already acquired data within the vertical array of registers 80 ofthe CCD until the host computer can resume accepting DMA transfers. Thisprocess proceeds until the last line of the region of interest istransmitted to the memory of the host computing system 12.

When the last line of the region of interest is read or shifted out ofthe horizontal register, the programmable imager controller 20 generatesa “fast shift” signal which instructs the CCD array 54 to dump allremaining data associated with the originally acquired image. This datapurge is accomplished in the manner described above. At this pointpreferably all of the vertical array of shift registers 80 are free ofcharge. All of the photosites 70 are also preferably empty since thetiming generator 20 pulses the CCD array 54 such that the array removesany charge from the photosites 70. In this way, the camera 16 is readyto receive another image acquisition command.

Once all DMA transfers of image data are completed, the host computingsystem 12 responds to the controller's 14 activation of one of theinterrupt lines on the host's I/O bus to trigger the processing of theinterrupt code, i.e. cam.VxD 137, that was loaded by the cam.exe program132 at boot time. The interrupt code 137 makes a call-back to memorymaintained by cam.dll 138 that stores the handle of the process thatclient.exe (or cam.exe) has most recently designated to receive noticethat a particular image buffer has been filled with new data.

According to another mode of operation referred to as “immediate mode,”,the image acquisition system of the invention can be used to accommodateboth image analysis and machine vision applications. In this mode ofoperation, data resulting from processing an earlier image dictates thenext action to be taken.

Accordingly, in this mode, the acquisition board 14 waits for a commandfrom the host to be written before initiating an image acquisition andprocessing cycle similar to that described above. Either client.exe 134or cam.exe 132 makes a call to cam.dll 138 to have the commandtransmitted. The arrival of this command causes the camera loader 42 totransmit the camera trigger signal 44 containing information from thecamera setup store 48 to the particular camera designated by that byte.The remaining steps of image acquisition and transfer proceed asdescribed above.

According to still another mode of operation, images can be taken atfixed time intervals from a selected camera. These time intervals toselected and can be any value greater than or equal to the minimum timecompatible with the selected exposure time and the time required totransfer the region of interest. This mode of operation is identical tothat described immediately above with the exception that the trigger forinitiating image acquisition by the selected camera is an associatedtimer forming part of the camera loader 42.

In this mode, either cam.exe 132 or client.exe 134 makes a call tocam.dll 138 that sends a command to the host computing system 12 and/orthe acquisition board 14. This command includes a particular targetcamera and a data word used to set the associated timer in the cameraloader 42. Every time the timer for the selected camera times out, thecamera loader 42 sends a camera trigger signal 44 to begin acquisitionby the selected camera.

In response, an image is returned from the selected camera 16 andtransferred to host memory 116, each time that the timer associated withthat camera times out.

According to yet another mode of operation, the image acquisition systemprovides for continuous image acquisition at a rate constrained only bythe size of the region of interest and the exposure time. This modebegins with a request by cam.exe 132 or client.exe 134 that particularcamera setup information be sent from the camera setup store 48 to aparticular camera 16 via the host interface of the acquisition board 14.The camera setup information causes the camera 16 to initiate anexposure and readout of image data, as set forth above in relation tothe description associated with FIGS. 1-3. The camera's response in thismode differs in that after exposure is complete, the programmable imagercontroller 20 does not immediately pulse the CCD array 54 to clear thephotosites of charge.

According to one preferred practice, if cam.dll 138 calculates that anexposure time longer than the readout time for the region of interest,then the camera setup information contained in the camera trigger signal44 causes the programmable imager controller 20 to begin the nextexposure immediately after the transfer of the image data. The foregoingpurging of charge from the photosites does not occur since the nextexposure taken by the camera 16 continues while the vertical array ofregisters 80 containing the image data from the previous exposure isread into the horizontal array of registers 82. The acquisition ofadditional frames continues beyond the end of the readout of thepreviously acquired data, until the appropriate exposure time isreached. At this time, the timing generator 20 commands a “frametransfer,” and begins both another exposure and another readout. Theprocess repeats until new camera setup information is transmitted to thecamera 16 from the camera setup store 48.

If, on the other hand, cam.dll 138 calculates a readout time for theregion of interest longer than the exposure time, then the camera setupinformation contained in the camera trigger signal 44 causes theprogrammable imager controller 20 to begin purging charge from thephotosites after a frame transfer and to continue to purge charge for atime interval as long as the excess of the readout-time overexposure-time. When the image transfer is completed, the programmableimager controller 20 initiates another “frame transfer,” resumes chargepurging, and begins another readout. This process continues until newcamera setup information is transmitted from the camera setup store 48to the camera 16.

In the event that FIFO status signal 79 is asserted for long enough tosubstantially affect the exposure during a cycle, the programmableimager controller 20 terminates the readout of image data for thatcycle. Readout resumes at the beginning of the region of interest forthe data then in the photosite region.

According to still another mode of operation, the system 10 canrecognize when a subject of interest is within a triggering region 95 inthe camera's field of view as shown in FIG. 9. Upon recognizing that asubject of interest has entered the triggering region 95, the systemautonomously triggers the capture of a region of interest 99, also shownin FIG. 9, within the cameras field of view and transmits the image datacorresponding to that region of interest to the system's memory and/orto the system's display. The size and location of the triggering region95 is independent of the size and location of the region of interest 99.This mode greatly facilitates automated image processing by allowing theuser or client software to adaptively control, based on the presence ofan object in the field of view, when image acquisition will occur.

This mode of operation is similar to the immediately preceding mode.Generally, the camera trigger signal 44 will include one or morethresholds which will be used by the acquisition board to determinewhether an image of the region of interest should be acquired. Thesethresholds are previously calculated by cam.dll 138 and loaded into thecamera setup store 48. In addition, the camera trigger signal 44 cancontain information necessary to define a triggering region in a mannersimilar to that used and already described for defining the region ofinterest.

In this mode, the trigger region is repeatedly captured and transmittedto the acquisition board 14 according to the procedure identified abovefor transmitting image data to the acquisition board. However, ratherthan being routed, as in the other modes, to the data FIFO 78, imagedata from the triggering region is sent to the proximity evaluator 88which accumulates the values of the pixels arriving from the triggeringregion. Upon completion of the transfer of the trigger region for agiven frame, the summary statistics of the pixels from the triggerregion, now stored in the proximity evaluator 88, is used to determinewhether the image from the region of interest should be captured. If thesystem determines that the accumulated value is such that the image fromthe region of interest should be captured, then that image is routed tothe data FIFO 78 as described earlier. If, on the other hand, the systemdetermines that the accumulated value is such that no image from theregion of interest should be captured, then another trigger region iscaptured, accumulated and compared.

Whether or not to acquire an image from the region of interest 95 basedon the image in the trigger region 99 can depend on the sum of thevalues of the pixels in the trigger region or on the deviation of thevalues of the pixels in the trigger region.

In one implementation of this mode, whether or not an image from theregion of interest is captured can depend on whether the accumulatedvalue of the pixels in the trigger region is above or below a threshold.In a second implementation of this mode, there can be n thresholds andthe capture of an image from the region of interest can be conditionedon which of the n+1 intervals defined by the n thresholds theaccumulated value of the pixels from the trigger region falls into. In athird implementation of this mode, the capture of an image from theregion of interest can be conditioned on whether or not the deviation ofthe summary statistics of the pixels from the trigger region from somenorm falls above or below a programmed deviation threshold. Furtherimplementations of this mode can be obtained by various booleancombinations of the above conditions.

The acquisition of an image from a region of interest can also beconditioned on the satisfaction of a temporal condition. For example,the system can be made to acquire an image from the region of interestonly when the time interval between the proposed acquisition and thelast acquisition is in excess of some temporal threshold. A temporalcondition such as this can be combined with the conditions onaccumulated pixel values and deviations as outlined above. Absent such afeature, an extended and homogenous object slowly traversing thecamera's field of view could result in multiple exposures of the sameobject.

In yet another mode of operation, an image from one camera operating inthe mode immediately above can trigger image acquisition and transfer byanother camera connected to the system.

In yet another mode of operation, the camera loader generates outputs,synchronized to each camera, to trigger eternal illumination systemssuch as flash units.

It will thus be seen that the invention efficiently attains the objectsset forth above, among those made apparent from the precedingdescription. Since certain changes may be made in the aboveconstructions without departing from the scope of the invention, it isintended that all matter contained in the above description or shown inthe accompanying drawings be interpreted as illustrative and not in alimiting sense.

It is also to be understood that the following claims are to cover allgeneric and specific features of the invention described herein, and allstatements of the scope of the invention which, as a matter of language,might be said to fall therebetween.

The invention claimed is:
 1. A system for transmitting images to amachine vision system having a host processor that allocates memory forstorage of said images acquired by said system, said system comprising:image acquisition means of a machine vision system including aphoto-sensitive element, the photo-sensitive element responsive to atleast one trigger signal asserted by external components locatedseparately from the image acquisition means for beginning acquisition ofone or more of said images in response to the at least one triggersignal, host memory means for storing data representative of a portionof one or more of said images, and image transfer means for controllingtransfer of one or more of said images from said image acquisition meansto said host memory means, said image transfer means having buffermemory associated with one or more data path levels for transferring aplurality of said images pending at least allocation of memory space insaid host memory means by the host processor for portions of saidimages, wherein said image acquisition means operates substantiallyindependently of the allocation of the memory space by the hostprocessor to acquire the plurality of the images at any time relative tothe allocation of the memory space in said host memory means.
 2. Asystem according to claim 1, wherein said buffer memory comprises a datafirst in, first out (FIFO) register, said data FIFO register beingcoupled to receive data representative of said images from said imageacquisition means and to release data representative of said images tothe host processor in response to a release signal.
 3. A systemaccording to claim 2, further comprising direct memory address datatransfer means for transferring said image data accumulated in said dataFIFO register directly to the memory space allocated in the host memorymeans, whereby said data transfer substantially reduces the number oftimes said system interrupts the host processor.
 4. A system accordingto claim 2, further comprising pre-processing means separate from thehost processor for receiving said image data and for at least partiallyprocessing said data without substantially interrupting the hostprocessor.
 5. A system according to claim 2, further comprisinginterrupt means coupled to said data FIFO register and to said imageacquisition means for interrupting said acquisition of said images whensaid data FIFO register is at least substantially full, therebypermitting reliable transfer of data corresponding to said images.
 6. Asystem according to claim 5, wherein said images are composed of aplurality of lines, and wherein said interrupt means is capable ofinterrupting said acquisition of said images between said linescorresponding to said images.
 7. A system according to claim 2, furthercomprising feedback means coupled between said image acquisition meansand said data FIFO register for monitoring a status of said data FIFOregister and for transferring an interrupt signal to said imageacquisition means to interrupt said acquisition of said images, saidfeedback means including interrupt means coupled to said data FIFOregister for generating said interrupt signal when said data FIFOregister is at least substantially full, whereby said interrupt meansimmediately and temporarily interrupts said acquisition of said images.8. A system according to claim 1, wherein said buffer memory comprises aplurality of registers, said plurality of registers being coupled toreceive data representative of said images from said image acquisitionmeans in response to said one or more trigger signals and to releasedata representative of said images to the host processor in response toa release signal.
 9. A system according to claim 1, wherein the at leastone trigger signal includes selected image acquisition parameters, andwherein said image acquisition means comprises: programmable controlmeans in circuit with said photo-sensitive element for generating saidat least one trigger signal and for programmably altering duringacquisition of said images said selected image acquisition parameters.10. A system according to claim 9, wherein said photo-sensitive elementincludes a solid state imaging array and said programmable control meansincludes a programmable imager controller.
 11. A system according toclaim 9, wherein said programmable control means comprises: a controlleradapted to be programmed with said selected image acquisition parametersand for generating said at least one trigger signal in response to aloading signal, and parameter loading means coupled to said controllerfor generating said loading signal and for receiving said imageacquisition parameters.
 12. A system according to claim 11, furthercomprising actuation means for generating an output actuation signal inresponse to an input command signal, said parameter loading meansgenerating said loading signal in response to said actuation signal, andparameter memory means, in communication with said parameter loadingmeans, for receiving said image acquisition parameters from the hostprocessor.
 13. A system according to claim 9, wherein said programmablecontrol means includes a programmable image controller circuit, saidcontroller circuit being adapted to be programmed with said imageacquisition parameters.
 14. A system according to claim 13, wherein saidprogrammable image controller circuit includes means for receiving saidimage acquisition parameters in less than or equal to about 2 μs.
 15. Asystem according to claim 13, wherein said programmable image controllercircuit includes means for reprogramming said circuit on the fly, inessentially real time.
 16. A system according to claim 13, wherein saidprogrammable image controller circuit operates at a driving frequency,said programmable image controller circuit comprising means forautomatically changing said driving frequency during acquisition of saidimages.
 17. A system according to claim 9, wherein said imageacquisition means further includes: actuation means for generating anoutput actuation signal in response to a command signal, parametermemory means in communication with said programmable control means forreceiving said image acquisition parameters from the host processor, anda camera loader circuit adapted to receive said selected imageacquisition parameters from said parameter memory means and said outputactuation signal, for programming said programmable control means withsaid image acquisition parameters.
 18. A system according to claim 9,wherein said buffer memory comprises one or more vertical registerscoupled to said photo-sensitive element for storing optical energyrepresentative of said acquired images, and one or more horizontalregisters positioned to receive at least a portion of said opticalenergy stored in said one or more vertical registers.
 19. A systemaccording to claim 18, wherein said photo-sensitive element is adaptedto acquire a region of interest corresponding to at least a portion ofsaid images, and wherein said image transfer means transfers a portionof said optical energy corresponding to a portion of said region ofinterest from said one or more vertical registers into said one or morehorizontal registers, the system further comprising: disabling means fordisabling transfer of said optical energy from said one or morehorizontal registers during said transfer of optical energy from saidone or more vertical registers into said one or more horizontalregisters, whereby said optical energy corresponding to said portion ofsaid region of interest is rapidly transferred from said one or morevertical registers to said one or more horizontal registers withoutactively removing said stored energy from said one or more horizontalregisters.
 20. A system according to claim 19, wherein saidphoto-sensitive element is adapted to acquire said region of interestcorresponding to at least a portion of said images, and wherein saidimage transfer means transfers a portion of said optical energy storedin said one or more vertical registers corresponding to a portion ofsaid region of interest into said one or more horizontal registers, thesystem further comprising: enabling means for enabling transfer of saidoptical energy stored in said one or more horizontal registerstherefrom, and disabling means for disabling transfer of said opticalenergy from said one or more vertical registers into said one or morehorizontal registers during said transfer of optical energy out of saidone or more horizontal registers.
 21. A system according to claim 9,wherein said images have a region of interest corresponding to a portionof said images, said region of interest having a selected number ofpixels, the system further comprising: pixel evaluation memory meanscoupled to the image acquisition means for storing a value correspondingto said selected number of pixels, and pre-determined pixel storagemeans coupled to said image acquisition means for storing a valuecorresponding to a predetermined number of pixels.
 22. A systemaccording to claim 21, further comprising: comparing means for comparingsaid value stored in said pixel evaluation memory means with said valuestored in said pre-determined pixel storage means, and means forgenerating a match signal when said values are equal, said programmablecontrol means including means for receiving said match signal and foraltering said image acquisition parameters.
 23. A system according toclaim 1, wherein said image acquisition means acquires said images inparallel with and substantially independently of the allocation of thememory space by the host processor.
 24. A system according to claim 1,wherein said images acquired by said image acquisition means includes aplurality of pixels, the system further comprising pixel correctionmeans in circuit with said image acquisition means for correcting saidpixels prior to transfer to said host memory means.
 25. A systemaccording to claim 24, wherein said pixel correction means comprises:second memory means for storing a plurality of pre-determined pixelcorrection values corresponding to each pixel of said images, andmultiplier means for multiplying each pixel of said images by saidcorresponding pixel correction value, thereby forming a corrected pixel.26. A system according to claim 24, wherein said pixel correction meansis separate from the host processor and performs said pixel correctionin real time.
 27. A system according to claim 1, wherein thephoto-sensitive element is adapted to acquire one or more of the imagesbased on a processing of a region of interest within a field of view ofthe photo-sensitive element.
 28. A system according to claim 27, whereinthe at least one trigger signal specifies the region of interest withinthe field of view of the photo-sensitive element.
 29. A system accordingto claim 27, wherein the image acquisition means transmits image datafrom within the region of interest to the host memory means and discardsimage data from outside the region of interest.
 30. A system accordingto claim 27, wherein the photo-sensitive element is adapted to acquireone or more of the images from another region of interest within thefield of view of the photo-sensitive element based on processing of theregion of interest within the field of view of the photo-sensitiveelement.
 31. A system according to claim 1, wherein the imageacquisition means selects the photo-sensitive element from a pluralityof photo-sensitive elements upon receiving the at least one triggersignal.